support for BLOB - #152
Open
julesyan wants to merge 11 commits into
Open
Conversation
… through an error message' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Collaborator
Author
|
Before this change, fetching a row with a large BLOB column would block until all the bytes were saved to disk, then send the response. Now the response goes back immediately with a blob_url, and the bytes are saved in the background. The client can fetch the URL whenever it's ready. |
julesyan
marked this pull request as draft
June 8, 2026 17:45
This was referenced Jun 8, 2026
Fixed getOutputParms() length bug and tracking bug for large BLOBs Cleaned up unreachable code Fixed Content-Length to use actual size Sanitize log injection for BlobServlet
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BLOB Support — Streaming via HTTP Token Endpoint
Overview
This PR adds full BLOB/BINARY support to mapepire, covering both reading BLOB columns from query results and writing BLOB values as prepared statement parameters.
The key design decision is how BLOB data is returned in daemon mode (WebSocket server). Rather than inlining potentially large binary payloads as Base64 in the JSON response — which would be bounded by the WebSocket message size limit — the server stores each BLOB value temporarily and returns a short-lived HTTP token URL instead. The client fetches the actual bytes via a separate GET /blob/{token} HTTP request. This means there is no upper bound on the size of a BLOB that can be transferred.
Daemon Mode — BLOB Output Shape
When a query returns a BLOB/BINARY column in daemon mode, the column value in the JSON response is not a string. Instead it is an object:
The client then fetches the raw bytes via:
The response is application/octet-stream — raw binary bytes, no encoding.
Token properties:
Single-use — consumed and deleted on first successful retrieval
TTL — expires after 60 seconds by default (configurable via BLOB_TOKEN_TTL env var)
Auth-protected — the Authorization header on the GET must match the credentials of the WebSocket connection that produced the token; mismatched credentials return 401 and discard the token
Memory/disk — BLOBs ≤ 1MB are held in heap; larger ones are spooled to a JVM temp file to bound memory pressure
Single Mode — BLOB Output Shape
In --single mode there is no HTTP server, so BLOB values fall back to inline Base64 — the same as the previous behaviour for clients using stdin/stdout.
BLOB Input (Parameters)
BLOB parameters in prepared statements are sent as Base64-encoded strings — the same as any other string parameter. The server detects the JDBC parameter type and automatically decodes the Base64 before calling setBytes() / setBinaryStream(). No special client-side encoding is needed beyond Base64.
I have attached three test files I used, the second and third one requires a table made from:
test-blob.txt
test-blob2.txt
test-blob-daemon.js
For testing blob deamon, start the deamon mode on any port, then run the command below on your machine to run